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TO ALL WHOM IT MAY CONCERN: 

BE IT KNOWN THAT we, MASAHIRO KONISHI, CHIAKI 
ICHIKAWA and HITOSHI YAMASHITA, all residing in Asaka-shi, 
Saitama, Japan, all citizens of Japan, have invented 
certain new and useful improvements in IMAGE SIGNAL 
PROCESSOR FOR PERFORMING IMAGE PROCESSING APPROPRIATE 
FOR AN OUTPUT DEVICE AND A METHOD THEREFOR of which the 
following is a specification- 



IMAGE SIGNAL PROCESSOR FOR PERFORMING 
IMAGE PROCESSING APPROPRIATE FOR AN OUTPUT DEVICE 

AND A METHOD THEREFOR 
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BAQKfiRQUN^^ILTHE_INVE^ 

^"'^^^::^;ZZr^.en.,on relate, ^o an ln,ag« s.«nal 
processor for processing ir.a«e signals obtained by Imaging a 
:::;ect .leia ^It. an ^agmg device , and a .et.od ot processing 

the image signals. 

some conventional cameras usrng srlverhalrde 
photosensitive tllms or digital still cameras have a tunctron 
so-called auto-brao.etlng. that continuo.slv Photographs a 
seguence of frames while gradually ■ shifting the exposur to 
thrierexposure side or overexposure side around the standard 
exposure of the camera. 

A camera using sllver-halide photosensitive films which 
performs auto-.racKeting is disclosed in ^^-^^^^^ ^l^^^ 
Lid-open Publication NO. 249534/1993. This camera can decrde 
the parameters , such as the number of frames to be photographed 
„ and the exposure correction amount, without recervrng 
instructions from the user. 

A camera that performs auto-bracKeting according to the 
p.lor art photographs a plurality of frames, meaning that the 
30 Photographed images differ in time. 

V, ^ n 9-second intervals each at one 
frames are photographed at 0 . 2 seconu x 

riven exposure levels, there is a I..-second -«-ence rn 
time between the first frame and the last frame. Therefore 
the movement of the object, the change in the photographing 
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image . 

to solve the problem. 
The present Invention seeks to solve t P 
assoclatea With the prior art .escribed above . Xt Is therefore 
an obiect of the present Invention to provide an l.a.e s.,nal 
Trooessor and also a method of processing an l.a.e sr.nal, 
:::: allows the nser to obtain a desired l.a.e d.rrn. 
auto-bracketing. 

X, is a specific object of the present Invention to 
provide an lma,e sl.nal processor that — 
of images which are different from each other rn color balance 
Iharpness, gradation characteristics and the like, whrch would 
not be achieved only by changing the exposure amount. 

TO solve the above problems, an Image signal processor 
for performing Image processing on a first image srgnal 
representative of an Image of a subject field captured by an 
imaging device to produce a second image signal comprises: a 
fZt memory for storing therein the first image srgnal, a 
Plurality of Image processors for each performing rmage 
prooe ing, different from each other, on the stored frrst 
llage signal to produce the second image signal different from 
Xther. and a second memory for storing therein the second 
image slg:als produced, wherein said plurality of rmage 
To essor's IncJde types and parameters of the Image proce .ng 
such that at least one of the types and parameters of the rmage 
processing are different between said plurality of rmage 



processors . 
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Mvantageously. t.a .ypes of i.a.e processing perio^ea 
by said plurality of l^age processors n,ay include at least one 
^liec J f.o™ a ,roup consisting of a c.an,e of .r...tnes 
a change of gradation change characteristics, a correctron of 
a cnauy saturation a change of a 

a color temperature, a change of saturation / 
contour , a change of a compression ratio and a change of a blac. 
:.el o the first image signal stored in said first memory 
the parameters of image processing .eing of the at least on 
^elected from the group, said plurality of image 
performing the image processing of the at least one select d 
from the group on the first image signal stored rn sard 
memory according to the parameters of image processing. 

in addition, in accordance with an aspect of the 
invention, each of said plurality of image P-«-«; 
correspond to one of a plurality of display un.ts whrch are 
;ro.ided for visualizing the image represented .y the second 
image signals stored in said second memory, each of sard 
plurality of image processors processing, according to the 
, p rameters, the second image signals to .e displayed on one . 
of the display units which corresponds to said image processor . 

in a further aspect of the invention, the image signal 
processor may further comprise a divider circuit for dividing 
5 the first image signal stored in said first memory mto a 
highlight area and a shadow area, each of said plural^y f 
image processors performing the image — ^j;^* 
least one of the types and the parameters of the rmage 
processing differs between the highlight area and the shadow 



30 area. 



Further in accordance with the present invention, an 
image signal processor for performing image 

first image signal representative of an image of a subject f reld 
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captured by an Imaging device to produce a second Image signal , 
comprl.es: a first memory for storing therein the first Image 
signal; a plurality of Image processors for each perfor^rng 
image processing, different from each other, on the stored 
first image signal to produce a third image signal different 
from each other; a second memory for storing therein the thxrd 
image signals produced, and an image composer circurt for 
composing the third image signals to produce the second image 
signal, wherein said plurality of Image processors Include 
types and parameters of the image processing such that at least 
one of the types and parameters of the image processing are 
different between said plurality of image processors. 

Still further in accordance with the present invention, 
a method of processing a first image signal representative of 
an image of a subject field captured by an imaging devxce to 
produce a second image signal comprises the steps of: storrng 
the first image signal in a memory; performing the same type 
of image processing on the stored first image signal accordrng 
, to parameters of image processing different from each other 
to produce the second image signals; and storing the produced 
second Image signals in a memory. 

ppTfU nR irBTPTTO H THF nRAWINGS 

5 The objects and features of the present invention will 

become more apparent from consideration of the following 
detailed description taken In conjunction with the 
accompanying drawings in which: 

FIG 1 is a schematic block diagram showing a preferred 
0 embodiment of an electronic still camera to which the present 
invention is applied; 

FIG 2 is a schematic block diagram showing an 
exemplified outline configuration of the signal processor of 
FIG- 1; 
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FIG 3 is a schematic block diagram showing an example 
of the ,ain adjusting circuit of the signal processor of FIG. 

„G 4 is a schematic block diagram showing an example 
of the usage-specific image processor of the signal processor 

"'figs. and 5B plot two types of processing performed 

by the gradation correction of FIG. 4; , „ i 

FIG 6 is a schematic block diagram, like FIG. l.showrng 

an alternative embodiment of an electronic still camera to 
which the present invention is applied; , , 

FIG 7 is a schematic block diagram, simxlar to FIG. 2, 
Showing an exemplified outline configuration of the signal 

''~:.\rss 80 Show examples of the areas extracted 

by the area extractor of FIG. 7; ^ vir 7 

FIG 9 is a schematic block diagram, srmrlar to FIG. 7, 
showing an exemplified outline configuration of the area 

extractor of FIG. 7; , ^„ 

FIG 10 is a histogram plotting pixels with respect to 
the brightness of a subjective field, useful for understanding 
how the area extractor of FIG. 7 divides an area; 

FIG 11 is a schematic block diagram, srrarlar to FIG. 
4, showing an example of the gain adjusting circuit of the 
signal processor of FIG. 6; and 

FIGS 12A and 12B plot conversion curves used by the 
radation correction of FIG. II to make brightness correctrcn. 
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gr 



Referring to the accompanying drawings, preferred 
embodiments of an image signal processor according to the 
p.esent invention will be described in detail. The xmage 
signal processor according to the present invention rs adapted 
to store a frame of image data formed by an imagrng devrce rnto 
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a buffer .e.o.y, set up control values of Image srgnal 
processing (l.age parameter Information, - -rrous 
situations for the fra.e of l.age data thus stored, and process 
the image signal for each of those situations to generate and 
store a plurality of frames of image data in a storage. Thrs 
processor therefore allocs various images, which meet varrous 
purposes, to be formed from one frame of photographed rmage 



data . 
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The image signal processor according to the present 
invention may be applied to a digital camera In thrs 
specification, a digital camera refers to a photographrng 
device, such as an electronic still camera and a drgrtal vrdeo 
camera, which photographs still Images or movie Images for 
recording them in the form of digital signals. 

The specific embodiments of the present Invention 
described are a digital camera which has a single^plate color 
filter provided Immediately In front of. or on. the 
photosensitive array and. especially, an electronic strll 
camera. Referring to FIGS. I-I2. the conf Iguratron and the 
operation of the electronic still camera accordrng to the 
present invention will be described. 

An electronic still camera in a preferred embodiment of 
the present invention can achieve a purpose simrlar to 
auto^bracKetmg according to the prior art by srmply 
photographing an object only once. The embodiment of the 
electronic still camera can also perform image processrng to 
, generate a plurality of different frames of image data that 
could not be done by changing the exposure amount only. 

in addition, the electronic still camera in the 
illustrative embodiment is adapted to generate image signals 
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in compliance with various types of image output system, more 
specifically a display unit, such as a CRT (Cathode-Ray Tube) , 
or a printer, by which photographed image signals are to be 
visualized in the form of pictures . In the description below , 
5 processing for generating image signals compliance with a type 
of output system is called "usage-specific image processing." 

In the illustrative embodiment, usage-specific image 
processing is performed on each of a plurality of frames of 

10 image data generated by image processing that is performed by 
changing image processing parameters . This processing , which 
includes processing corresponding to the auto-bracketing 
function according to the prior art, is called "gain 
adjustment." It should be noted that gain adjustment and 

15 usage-specific image processing are independent of each other . 
One of them may be executed independently. 

As shown in FIG. 1, an electronic still camera 10 in the 
illustrative embodiment comprises a camera proper 11 and a 

20 detachable memory card attachment 20 connected together via 
a connector 30. The camera proper 11 comprises an optical 
system 11a, a color filter lib, a CCD (Charge-Coupled Device) 
image sensor 11c, a preprocessor lid, an analog- to-digital 
(A/D) converter lie, a signal processor 12, a data compressor 

25 llf , an input/output interface llg, a data input unit llh, and 
a system controller 13 , which are interconnected as shown. The 
memory card attachment 2 0 comprises an input/output interface 
21 and a memory card 22, interconnected as shown. 

30 ■ The optical system 11a comprises a photographing lens 

110a, an iris and components associated therewith, not shown 
in the figure. The color filter lib is a single-plate 
including color filter segments formed in a filter array as 
described above. The electronic still camera 10 specifically 
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includes a Bayer color filter in the embodiment . The invention 
is not restricted to that specific filter array but applicable 
to other types of color filter arrays. In the description 
below, three primary colors are represented by R (red) , G 
(green) and B (blue) . The present invention may be applied 
regardless of the color filter arrangement type. This color 
filter array lib is provided immediately before or on the 
photosensitive array of the CCD image sensor 11c. 

On the CCD image sensor 11c, the photosensitive cells, 
or photodiodes, of the CCD image sensor 11c corresponding in 
position to the filter segments of the color filter lib are 
provided to form an array of pixels. When incident light from 
the subject field falls on those cells, the cells photo- 
electrically convert the incident light into electric charges. 
The CCD image sensor 11c outputs those charges in the form of 
image signal, corresponding to the pixels, to the preprocessor 
lid. 

The preprocessor lid is adapted to amplify the supplied 
image signal to a desired level and outputs the amplified signal 
to the A/D converter lie. The A/D converter lie is adapted 
to convert the analog signal , output from the preprocessor lid, 
to the digital signal. 

The signal processor 12 is adapted for performing color 
correction on the digital signal thus received, storing the 
color-corrected digital signal temporarily into the buffer 
memory 12b, FIG. 2, and performing the image processing, such 
as gain adjusting and usage-specific image processing, on the 
stored digital signal to generate a plurality of frames of image 
data. The signal processor 12 may comprise a special-purpose 
IC (Integrated Circuit), a general-purpose CPU (Central 
Processor Unit) or microprocessor, or a DSP (Digital Signal 



Processor) . It should be noted that the IC, CPU, or DSP may 
comprise not only the signal processor 12 but also the function 
of the data compressor llf which will be described later. 

The signal processor 12, which is one entity, has a 
plurality of image processing features to perform various types 
of processing on the image signal obtained from the image sensor 
11c and stored to generate processed image signals which are 
different from each other. The signal processor 12 performs 
a plurality of types of image processing where at least one 
of the image processing types and image processing parameters 
is different from others. A plurality of signal processors, 
each corresponding to one of the plurality of image processing 
features, may be provided. 

As shown in FIG. 2, the signal processor 12 comprises 
a color interpolation circuit (RGB interpolation circuit) 12a, 
a buffer memory 12b, a gain adjusting circuit 12c, and a 
usage-specific processor 12d. The color interpolation 
circuit 12a, buffer memory 12b, gain adjusting circuit 12c, 
and usage-specific processor 12d perform processing in 
response to the control signal from the system controller 13. 
The control signal contains the timing signal, as well as 
instructions and data required for processing. 

The color interpolation circuit 12a is adapted to 
generate color data not obtainable directly by the actual 
photosensitive cells included in the CCD image sensor 11c, by 
interpolating the color data of the surrounding, actual cells. 
Thus, at each time of photographing, the color interpolation 
circuit 12a outputs the R data, G data, and B data at the 
respective pixel positions to store them into the buffer memory 
12b. 



The buffer memory 12b is adapted to store therein image 
data supplied from the color interpolation circuit 12a. The 
memory is large enough, for example, to contain one frame of 
image data. Image data is input to, and output from, the buffer 
5 memory 12b in response to an instruction from the system 
controller 13. Image data is stored in the buffer memory 12b 
each time an image of the object is photographed. In response 
to each instruction from the system controller 13, one frame 
of image data is output repeatedly from the buffer memory 12b 
10 to the gain adjusting circuit 12c. 

The gain adjusting circuit 12c is adapted to perform gain 
adjustment on the image data read out from the buffer memory 
12b. More specifically, in response to the instruction 
15 received from the system controller 13 and indicating how many 
times gain adjustment is to be executed and the exposure 
correction amount (parameter information) , the gain adjusting 
circuit 12c multiples the exposure value of image data from 
the buffer memory 12b by a predetermined value. 

20 

For example, when five frames of image data are to be 
generated during gain adjusting and when the exposure 
correction amount is to be incremented in steps of 0 . 5EV 
(Exposure Value) , the system controller 13 issues the 

25 instruction described below. The system controller 13 
sequentially generates the pixel addresses included in one 
frame of image data and sends them to the buffer memory 12b. 
The system controller 13 repeats this processing five times. 
At the same time, the system controller 13 sends exposure 

30 correction values, e.g. -1.0, -0.5, 0.0, +0.5, and +1.0, to 
correct the exposure value (EV) each time it sends one frame 
of image data to the memory buffer. For example, the value 
-1 . 0 means that -1 . OEV of exposure correction will be executed. 
In response to this instruction, the gain adjusting circuit 
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12c performs processing associated with this exposure 
correction. More specifically, it multiples the exposure 
value of the image data from the buffer memory 12b by a value 
1/2. The values of -1.0, -0.5, 0.0, +0.5, and +1.0 mean that 
the gain adjusting circuit 12c will multiply the exposure value 
of the image data by values 1/2, l/2^^^ 1.0, 2"\ and 2, 
respectively. It should be noted that those numeric values 
are examples only but that exposure correction values are not 
limited to those specific values. 

The usage-specific processor 12d is adapted to receive 
image data from the gain adjusting circuit 12c and perform image 
processing according to the information (parameter 
information) concerning a display unit specified by the system 
controller 13. Image data processing may be appropriate for 
two types, for example, one for CRT (Cathode Ray Tube) display 
and the other for printer output (hardcopy) of data, each having 
its own image quality. In the description below, those two 
types of processing are called CRT processing and hardcopy 
processing, respectively. The two types of processing will 
be described later in detail. The usage-specific processor 
12d is also adapted to convert RGB data into luminance data 
Y and color difference or chrominance data B-Y and R-Y . 
Although only one processing setting is shown for each of CRT 
processing and hardcopy processing in the description of the 
embodiment, the present invention is not limited to the one 
setting. Two or more settings may be specified depending upon 
the type of the CRT or the printer. The present invention may 
also be applied to display systems other than the CRT, such 
as a liquid crystal display, an EL (Electro-Luminescence) 
display, and a plasma display. 

The signal processor 12 outputs the luminance data Y and 
the color difference data B-Y and R-Y to the data compressor 
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llf shown in FIG. 1. The data compressor llf is adapted to 
compress the luminance data Y and the color difference data 
B-Y and R-Y by reducing the number of bits of the data using 
Huffman coding or differential PCM as the compression method. 
5 The data compressor llf uses one of these compression methods 
to reduce the data amount and then outputs the reduced data 
to the input/output interface llg. 

The data compressor llf can compress image data using 
10 the compression ratio in accordance with the display systems 
specified by the system controller 13. The compressor uses 
a higher compression ratio for image data to which the image 
quality for CRT display was assigned by the usage-specific 
processor 12d, and a lower compression ratio for image data 
15 to which the image quality for printer output was assigned. 
The processing of the data compressor llf will be described 
later in detail. 

The CCD image sensor 11c, preprocessor lid, A/D converter 
20 lie, signal processor 12, and data compressor llf are timed 
with each other in response to the timing signal output from 
the system controller 13. The system controller 13 contains 
a system controller for controlling the constituent elements 
included in the whole electronic still camera 10, a timing 
25 signal generator for generating the timing signal, and an 
address controller, not shown in the figure. The system 
controller 13 also controls the timing signal generator and 
the address controller to control the whole electronic still 
camera 10. The timing signal generator, under the control of 
30 the system controller 13, generates the signals which are used 
for driving the whole electronic still camera 10 and adjusting 
the operational timing, and supplies those signals to the 
components as shown in FIG. 1. 



12 



The address controller outputs address data to the buffer 
memory 12b to sequentially input image data to, and output image 
data from, the buffer memory 12b. In addition, to store 
processed image data into the memory card 22, the system 
controller 13 outputs required data and addresses to the memory 
card 22 via the input/output interface llg and the connector 
30 of the camera proper 11 and via the input/output interface 
21 of the memory card attachment 20. The required data 
includes a file name necessary for image data input/output , 
a photographing date and time , a photographing condition 
(exposure value) , and an image processing condition (gain 
adjusting condition, display system information) . 

The data input unit llh includes components for user 
operation such as a mode selector button, a shutter release 
button, and a liquid crystal monitor, not shown, on which the 
mode of the electronic still camera 10 is displayed. When the 
user operates the mode selector button or the shutter release 
button, the data input unit llh outputs operational result data 
to the system controller 13. 

User instructions entered by the mode selector button 
include whether to adjust the gain, whether to allow, if the 
gain is to be adjusted, the camera 10 to automatically decide 
the number of frames to be shot and the exposure correction 
amount, the number of frames to be shot and the exposure 
correction amount if manually decided by the user, and whether 
to perform usage-specific image processing. 

In response to an instruction indicating that, during 
gain adjusting, the electronic still camera 10 will 
automatically decide the number of frames and the exposure 
correction amount, the system controller 13 may decide the 
proper exposure value, the number of frames, and exposure 



correction amount according to the method disclosed in Japanese 
Patent Laid-Open Publication No. 249534/1993. 

The signal processor 12, adapted for processing digital 
5 signals, may be implemented in the form of dedicated digital 
signal processing IC. Because the primary operational 
function of the processor is multiplication and addition, a 
general-purpose DSP (Digital Signal Processor) may also be 
applicable . 

10 

Next, referring to FIGS. 3 and 4 showing the detailed 
configurations of the gain adjusting circuit 12c and the 
usage-specific processor 12d, respectively, the signal 
processor 12 with the components described above will be 

15 described more in detail. As shown in FIG. 3, the gain 
adjusting circuit 12c comprises a multiplier 121 and a gain 
setter 122. The multiplier 121 is connected to receive image 
data from the buffer memory 12b. It is assumed that image data 
is sent from the buffer memory 12b to the gain adjusting circuit 

20 12c in the raster scanning fashion. That is, after one line 
of pixel data is output horizontally, image data on the next 
line is output. One pixel of data, containing 24 bits in length 
(8 bits for each of R, G, and B) , is sent over eight signal 
lines, i.e. eight bits in parallel. 

25 

The gain setter 122 is connected to receive an exposure 
correction amount instruction from the system control 13 . For 
example, when the instruction indicating the value of "-1.0" 
is received from the system control 13, the gain setter 122 
30 then sets the gain of the multiplier 121 to 1/2. Then, the 
multiplier 121 multiples the exposure value of the received 
image data by 1/2 and outputs the result to the gain adjusting 
circuit 12c. When no gain adjusting is performed, the system 
controller 13 sends the instruction indicating the value of 
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"0.0" to the gain setter 122. This instruction means that 
image data from the buffer memory 12b is sent to the 
usage-specific processor 12d without making any change in the 
gain adjusting circuit 12c. 

5 

FIG. 4 shows the usage-specific processor 12d which is 
connected to receive image data from the gain adjusting circuit 
12c. The usage-specific processor 12d comprises a selector 
123, a gain control 124, a gradation control 125 comprising 
10 LUTs (Look Up Tables) , a luminance-chrominance matrix 126, an 
edge or contour enhancement circuit 127, a saturation 
emphasizing circuit 128, and a parameter setter 129, 
interconnected as shown in the figure. 

15 The selector 123 distributes RGB data, received 

sequentially from the gain adjusting circuit 12c, into R data, 
G data, and B data. This distribution allows processing for 
each color to be executed independently if necessary. 

20 The gain control 124 amplifies each color data 

distributed by the selector 123. More specifically, the gain 
control 124 multiples color data by a numeric value 
predetermined for the display system (CRT or hardcopy) . 

25 The gradation control 125 performs gradation conversion, 

that is, processing such as gamma correction. More 
specifically, the gradation control 125 includes a gradation 
correction curve defining the correspondence of the value of 
color data to be output with respect to the value of received 

30 color data, and converts the received color data according to 
the correction curve. FIGS. 5A and 5B show some examples of 
gradation correction curves. FIGS. 5A and 5B show the 
gradation correction curves to be used for CRT processing and 
hardcopy processing, respectively. In FIGS. 5A and 5B , the 
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horizontal axis indicates the value of entered color data, and 
the vertical axis indicates the output value of color data 
corresponding to the received color data. 

5 These curves are employed for the reasons described below. 

The correction curve in FIG. 5A is used for a CRT because the 
photoelectric conversion characteristics of the CRT screen is 
usually 2 . 2th power of the input so that conversion using 0 . 45th 
power of the input (0.45 is the reciprocal of 2 . 2 ) is performed. 
10 Using this correction curve makes the contrast relatively lower 
than the contrast before gradation correction. Therefore, 
this type of correction is called "contrast decreasing". 

The gradation correction curve in FIG. 5B is used for 
15 a hardcopy because the S-shaped characteristics are desirable 
when restoring the contrast of image data on a hardcopy. That 
is, since it is usually important to retain definite contrast, 
or intensity difference, on a hardcopy, a sharper contrast is 
desired. The intermediate portion of the S-shaped curve has 
20 a slope corresponding to gamma==l . This portion corresponds 
to the flesh color , and gamma=l is desirable for the flesh color . 
Using this correction curve makes the contrast relatively 
higher than the contrast before corrected in gradation. 
Therefore, this type of gradation correction is called 
25 "contrast increasing". 

It is also possible for the gain control 124 and the 
gradation control 127 to use a gain and a gamma correction curve 
which are different in dependent upon colors of image data. 

30 

The luminance-chrominance matrix 12 6 is adapted to 
perform matrix operation to convert RGB data to the luminance 
signal (Y) and color difference signal (Cr, Cb) . The matrix 
operation is performed using the expressions read below. 
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Yl = Kll X Rl + K12 X Gl + K13 x Bl , 
Crl = K21 X Rl + K22 x Gl + K23 x Bl , (D 
CBl = K31 X Rl + K32 x Gl + K33 x Bl , 

where, Rl , Gl , and Bl are color data that is input to 
the luminance-chrominance matrix 126 . They represent the red , 
green and blue components, respectively. Kll, K12, K13, K21, 
K22, K23, K31, K32, and K33 are correct coefficients. Yl is 
the luminance signal that is output by the luminance- 
chrominance matrix 126. Crl and Cbl are color difference 
signals output by the luminance-chrominance matrix 126. The 
luminance signal Yl is sent to the edge enhancing circuit 127, 
and the color difference signals Crl and Cbl are sent to the 
saturation emphasizing circuit 128. 

The edge enhancing circuit 127 emphasizes the edge or 
contour of image data (aperture) to increase the contrast of 
the whole image. A known method may be used for emphasizing 
the contour of a picture. For example, to emphasize a 
horizontal contour line, the method described below is 
performed. The original signal YO is delayed by one horizontal 
scan period of time IH into a iH-delayed signal YH , and by two 
horizontal scan periods of time 2H into a 2H-delayed signal 
Y2H. In this case, YH+K x [ YH- (Y0+Y2H) /2 ] is the luminance 
signal in which the horizontal contour line is emphasized, 
where K is a constant defining the degree of emphasizing the 
contour line. The larger the value K, the more emphasized the 
contour . This processing may be performed by a low-pass filter 
(LPF) . 

A similar known method may be used for emphasizing a 
vertical contour line. The original signal YO is delayed by 
the predetermined delay time Iz into a delayed signal, and 



by two times of the delay time, i.e. 2z into a delayed signal 
Y2 T . In this case , Y t +K x [Y t - (Y0+Y2 t ) /2 ] is the luminance 
signal in which the vertical contour line is emphasized. 

The saturation emphasizing circuit 12 8 is adapted to 
modify the color difference signal to convert saturation. 
More specifically, the circuit 128 multiples a numeric value 
predetermined according to the display system by the color 
difference signal. More specifically, the saturation is 
represented by (Cr**2+Cb**2) ^''^ To emphasize the saturation , 
the circuit 128 multiples the color difference signals, Cr and 
Cb, by a predetermined numeric value. The same predetermined 
value, or different predetermined values, may be used for the 
color difference signal Cr and the color difference signal Cb . 
The predetermined numeric value should be decided considering 
the hue . 

Table 1 shows in the column of usage-specific image 
processing how the gain control 124, gradation control (LUTs) 
125, luminance-chrominance matrix 126, edge enhancing circuit 
127, and saturation emphasizing circuit 128 convert image data 
for the specific display systems during the CRT processing and 
the hardcopy processing described above. 





Usage-specific image processing 


CRT processing 


Hardcopy processing 


Gain 


Large 


Small 


LUT 


A 


B 


Matrix coefficient 


Change 


Change 


Contour enhancing 


Strong 


Weak 


Saturation emphasis 


Strong 


Weak 


Compression ratio 


High 


Low 
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The row Gain in Table 1 represents the gain control of 
the gain control 124, and the row LUT represents the gradation 
correction of the gradation control 125. Note that the letter 
"A" in this row indicates the gradation correction curve 
plotted in FIG. 5a and that "B" indicates the correction curve 
shown in FIG. 5B . The row Matrix coefficient represents the 
conversion of the luminance-chrominance matrix 126, and the 
rows Contour enhancing and the Saturation emphasis represent 
the conversion of the edge enhancing circuit 127 and the 
saturation emphasizing circuit 128 . The row Compression ratio 
represents the conversion of the data compressor llf shown in 
FIG. 1. 

The reason for the settings shown in Table 1 will be 
described. In general, when comparing the color range which 
a CRT can represent with the color range which a hardcopy 
(especially a printer) can represent, the color range which 
the CRT can represent is wider. On a CRT, a brighter image 
looks clearer. On a hardcopy, a darker image looks clearer. 

For that reason, on a CRT the gain and the saturation 
emphasis are increased whereas on a hard copy the gain and the 
saturation emphasis are decreased. Also, the contour 
enhancement is increased on a CRT because an contour-enhanced, 
sharp image is desirable. On the other hand, the contour 
enhancement is decreased on a hardcopy because a soft image 
is desirable. The reason for the LUT setting was described 
with reference to the gradation control 125. The compression 
ratio will be descried later. 

In response to the signal indicating the type of display 
system from the system controller 13, the parameter setter 129 
shown in FIG. 4 sets the value in the components of the 
usage-specific processor 12d according to the setting shown 



in Table 1. More specifically, for the gain control 124, the 
parameter setter 129 has two types of coefficients 
(corresponding to the gains of the amplifier) for two types 
of display systems. In response to the instruction from the 
5 system control 13, the parameter setter 129 outputs one of two 
types of coefficients to the gain control 124. The gain 
control 124 multiples color data by the coefficients. 

The parameter setter 129 performs the same setting for 
10 the saturation emphasizing circuit 128. 

To the gradation control 125, the parameter setter 129 
outputs the signal which specifies a gradation correction curve . 
The setter 129 also outputs the above-described coefficients, 
15 which specify to which extent the contour is to be emphasized, 
to the edge enhancing circuit 127. 

The image data, processed by the usage-specific 
processor 12d of the signal processor 12 as described above, 

20 is sent to the data compressor llf shown in FIG. 1 . In response 
to an instruction from the system controller 13 indicating 
whether to increase or decrease the compression ratio, the data 
compressor llf compresses the received image data according 
to the instruction. The image data may be compressed, for 

25 example, based on the JPEG (Joint Photographic coding Experts 
Group) standard. In the JPEG standard, image data is 
compressed by sequentially performing DCT (Discrete Cosine 
Transform) processing, quantization, and coding (run length 
coding and Huffman coding) . To change the compression ratio, 

30 the quantization table used for quantization should be changed. 
Specifically, two tables are prepared, one for higher 
compression and the other for lower compression. The image 
data is quantized using either of the two quantization tables 
in dependent upon the specified compression ratio. 
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The reason for setting the compression ratio, as shown 
in Table 1 in accordance with the display system, is that a 
higher compression ratio is selected for a wider range image 
data to be used for CRT processing whereas a lower compression 
ratio for a narrower range image data to be used for hardcopy 
processing. This setting allows one frame of image data to 
be stored in the same memory capacity. 

The compressed image data is sent to the memory card 22 
and stored therein. The memory card may be a semiconductor 
memory (flash memory, and the like) , a magnetic disk, or an 
optical disc. 

In operation, the user manipulates the data input unit 
llh to specify whether to perform gain adjusting and 
usage-specific image processing. When the user wants to do 
gain adjusting, he or she must specify the number of image data 
frames to be created and the exposure correction amount. In 
response to the user instruction, the system controller 13 
decides whether to perform gain adjusting and/or usage- 
specific image processing. When both gain adjusting and 
usage-specific image processing are to be performed, the system 
controller 13 causes the signal processor 12 to do its 
processing the number of times equal to two times of the number 
of processing cases during gain adjusting. In the description 
below, assume that the user has specified that both gain 
adjusting and usage-specific image processing are to be 
performed, that the number of processing cases during gain 
adjusting is three, and that the exposure correction amount 
is incremented in steps of lEV. In this case, six frames of 
image data will be finally obtained. 

When the user depresses the shutter release button after 
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specifying the processing mode setting, the imaging device 11c 
captures an image of a subject. The obtained image data is 
processed by the preprocessor lid and the A/D converter lie 
and then sent to the signal processor 12. The image data, for 
which color interpolation is performed by the color 
interpolation circuit 12a of the signal processor 12, is in 
turn stored in the buffer memory 12b. Each time an output 
instruction and an address are received from the system 
controller 13 , the image data is sent from the buffer memory 
12b to the gain adjusting circuit 12c. In response to the 
instruction from the system controller 13, the gain adjusting 
circuit 12c seguentially shifts the coefficients 1/2 , 1.0, 2.0, 
1/2, 1.0, 2.0, by which the image data is to be multiplied. 

The image data for which gain-adjusting has been 
performed is sent to the usage-specific processor 12d. The 
usage-specific processor 12d performs CRT processing for the 
first three frames of image data, and hardcopy processing for 
the last three frames of image data, according to the settings 
in Table 1 . 

After usage-specific image processing, the first three 
frames of image data are compressed at a higher compression 
ratio, and the last three frames of image data at a lower 
compression ratio. Then, the compressed data is sent to the 
memory card, one frame of data at a time. 

Unlike a camera that performs auto-bracketing according 
to the prior art, the electronic still camera according to the 
present invention photographs an image only once. Therefore, 
a problem that the photographing timing differs with 
photographed images does not arise. As a result, the user may 
obtain a desired image without concerning the movement of an 
object, changing image-shooting conditions, or unintentional 



movement of the operator's hands during photographing. 

From the six processing options listed in Table 1, i.e. 
the gain, LUT , matrix coefficients, contour enhancement, 
5 saturation emphasis, and compression ratio, one to five 
processing methods may be selected and combined. 

Next, an alternative embodiment of the present invention 
will be described. In this embodiment, white balance 
10 correction and brightness correction are made as gain adjusting 
in a plurality of ways. While balance correction is made by 
three adjustments, that is, gain adjustment, matrix 
coefficient adjustment, and saturation adjustment. 

15 Gradation correction is performed for brightness 

correction. To do so, in this embodiment one frame of image 
is dealt with as two parts according to the amount of light: 
a bright part where light falls (highlight area) and a shadowy 
part where little or no light falls (shadow area) . Different 

2 0 parameters are used for the highlight area and the shadow area. 
This processing will be described later in detail. 

One or both of white balance correction and brightness 
correction may be made for one frame of image data. The user 
25 may specify how correction will be made. 

In this embodiment, black level offsetting is also done. 
The black level refers to the status in which the gradation 
value is null. Black level offsetting refers to the increase 
30 or decrease of the gradation value of the whole image by a 
predetermined amount. The purpose of black level offsetting 
is to decrease the gradation value when the whole image looks 
too whitish due to a halation or to increase the gradation value 
to brighten the whole area of the image photographed at night. 
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Black level offsetting is done by adding or subtracting 
a predetermined value to or from the whole image data using 
an adder. 

Black level offsetting may also be done in the first 
embodiment. In the first embodiment, an adder may be added 
before or after the multiplier of the gain adjusting circuit 
12c to offset the black level. 

The instant embodiment is similar to the first embodiment 
except for the configuration and the operation of the signal 
processor, system controller, and data input unit. The 
remaining components , i.e. the optical system 11a , color filter 
array lib, CCD image sensor 11c, preprocessor lid, A/D 
converter lie, data compressor llf and input/output interface 
llg of the camera proper 11, and the input/output interface 
21 and memory card 22 of the memory card attachment 20, perform 
the same processing as those of the first embodiment. 

Therefore, the alternative embodiment will be described 
below with emphasis on the signal processor, system controller, 
and data input unit. The same reference numbers denote the 
components having the function similar to that in the first 
embodiment. The description of such components is omitted. 

As shown in FIG. 6, an electronic still camera lOA in 
the alternative embodiment comprises a camera proper llA and 
the memory card attachment 2 0 connected together via the 
connector 30. The camera proper llA comprises the optical 
system 11a, the color filter array lib, the CCD image sensor 
11c, the preprocessor lid, the A/D converter lie, a signal 
processor 212, the data compressor llf, the input/output 
interface llg, a data input unit llhA, and a system controller 
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13A. The memory card attachment 20 comprises the input/output 
interface 21 and the memory card 22. 

The signal processor 212 makes color correction in 
5 response to the digital signal from the A/D converter lie, 
temporarily stores the color-corrected digital signal into the 
buffer memory, and performs image processing (gain adjusting) 
on the stored digital signal to generate a plurality of frames 
of image data. 

10 

As shown in FIG. 7, the signal processor 212 comprises 
the color interpolation circuit 12a, the buffer memory 12b, 
an area extractor 212a, an area storage 212b, and a gain 
adjusting circuit 212c. The color interpolation circuit 12a, 

15 buff er memory 12b, area extractor 212a, area storage 212b, and 
gain adjusting circuit 212c are adapted to perform the 
processing in response to the control signal from the system 
controller 13A. The control signal contains the timing signal , 
as well as instructions and data required for processing that 

20 will be described later. 

The buffer memory 12b stores therein image data supplied 
from the color interpolation circuit 12a. The memory 12b is 
large enough, for example, to contain one frame of image data. 
2 5 Image data is input to, and output from, the buffer memory 12b 
in response to an instruction from the system controller 13A. 
Image data is stored in the buffer memory 12b each time an image 
is photographed. 

30 When brightness correction is made, one frame of image 

data is output first from the buffer memory 12b to the area 
extractor 212a in response to an instruction from the system 
control 13A. The area extractor 212a extracts the highlight 
area and the shadow area. This extraction will be described 
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with reference to FIGS. 8A, 8B and 8C. FIG. BA shows an image 
photographed outdoors on a fine day under the daylight against 
the sun or against a bright background. This photograph shows 
the highlight area and the shadow area most clearly. 

5 

One frame of image shown in FIG. BA comprises shadowy 
part (shadow area) A and highlight area B which is the fair 
sky, clouds, or a bright background. This image, where the 
shadow is dark, gives a sharp impression. By increasing the 

10 brightness of shadowy part A and decreasing the brightness of 
bright part B, the shadow is decreased and the image looks 
softer. To process an image whose brightness varies in 
dependent upon the parts of the image, the shadowy part shown 
in FIG. SB and the bright part shown in FIG. 80 are extracted 

15 from the one frame of original image. 

More specifically, a check is made, during extraction 
processing, for each pixel to see if it is a highlight or a 
shadow. The binary value "1" is output to the area storage 

20 212b when the pixel is a highlight; the binary value "0" is 
output to the area storage 212b when the pixel is a shadow. 
In the description below, those values "^1" and "0" are called 
area information. The area information is used by the gain 
adjusting circuit 212c to perform different processing for the 

25 highlight area and the shadow area. 

The area storage 212b receives the area information from 
the area extractor 212a and stores therein the area inf ormation 
on the pixels included in one image. In response to an 
30 instruction from the system controller 13A, this information 
is sent to the gain adjusting circuit 212c. 

To do brightness correction, the image data is sent from 
the buffer memory 12b to the gain adjusting circuit 212c after 



26 



the area information is stored in the area storage 212b. The 
image data is sent each time an instruction is sent from the 
system controller 13A to the buffer memory 12b. The number 
of times the image data is sent equals the number of times white 
5 balance correction parameters are to be changed multiplied with 
the number of times brightness correction parameters are to 
be changed. 

When only white balance correction is to be made, the 
10 area extractor 212a and the area storage 212b are not active. 

The gain adjusting circuit 212c is adapted for receiving 
the image data from the buffer memory 12b, and adjust the gain 
of the received image data. More specifically, the gain 

15 adjusting circuit 212c processes the image data from the buffer 
memory 12b according to the instruction from the system 
controller 13A. This instruction indicates whether to make 
white balance correction, whether to make brightness 
correction, the number of times correction parameters are to 

2 0 be changed, or the parameter values (color temperature or 
exposure amount) to be used. 

Assume that both white balance correction and brightness 
correction are to be made, that the white balance correction 
25 parameters are to be changed three times, and that the 
brightness correction parameters are to be changed twice. 
Then, the system controller 13A issues the instruction 
described below. 

30 First, in order to create area information, the system 

controller 13A sequentially generates the addresses of the 
pixels included in one frame of image data and sends them to 
the buffer memory 12b. At the same time, the system controller 
13A tells the area extractor 212a to receive image data from 
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the buffer memory 12b to do the area extraction. 

After the area extractor 212a performs the area 
extraction to store area information in the area storage, the 
5 system controller 13A sequentially generates the addresses of 
the pixels included in one frame of image data and sends them 
to the buffer memory 12b. The system controller ISA repeats 
this processing six (=3x2) times. At the same time, each 
time the system controller 13A performs this processing, it 
10 sends an instruction to the gain adjusting circuit 212c, 
indicating whether to make white balance correction and/or 
brightness correction as well as parameter information (color 
temperature and exposure amount) to be used during these 
corrections . 

15 

The gain adjusting circuit 212c also converts RGB data 
to luminance data Y and color difference data B-Y and R-Y. The 
gain adjusting circuit 212c outputs the luminance data Y and 
the color difference data B-Y and R-Y to the data compressor 
20 llf shown in FIG. 6. 

The gain adjusting circuit 212c is also adapted to 
receive from the system controller 13A an instruction 
indicating whether to perform the black level offsetting, and 
25 perform the black level offsetting accordingly. The 
instruction indicating whether to perform the black level 
offsetting is sent to the system controller 13A whenever the 
user presses the mode selector button which will be described 
later. 

30 

Upon receiving a black level offsetting instruction, the 
system controller 13A checks the pixels stored in the buffer 
memory 12b or the gradation values of the sampled pixels to 
determine whether or not the black level offsetting is to be 
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performed. For example, when the number of black level pixels 
is smaller than a predetermined number and when the number of 
white level pixels is larger than a predetermined number, the 
system controller 13A determines that there is a halation in 
the image. Then, the system controller 13A tells the gain 
adjusting circuit 212c to decrease the gradation values of the 
pixels. Conversely, when the number of black level pixels is 
larger than a predetermined number, the system controller 13A 
determines that the whole screen is too dark, for example, 
because the image was photographed at night. Then the system 
controller ISA instructs the gain adjusting circuit 212c to 
increase the gradation values of the pixels. 

When the system controller 13A receives a black level 
offsetting instruction, it may generate image data for which 
the black level offsetting is performed, as well as image data 
for which no black level offsetting is performed, at the same 
time . 

The CCD image sensor lie, preprocessor lid, A/D converter 
lie, signal processor 212, and data compressor llf are timed 
with each other in response to the timing signal output from 
the system controller ISA. The system controller ISA contains 
a system controller for controlling the whole electronic still 
camera lOA, a timing signal generator for generating the timing 
signal, and an address controller, not shown in the figure. 
The system controller ISA also controls the timing signal 
generator and the address controller to control the whole 
electronic still camera lOA. The timing signal generator, 
under the control of the system controller, generates the 
signal driving the whole electronic still camera lOA and the 
operational timing adjusting signal and supplies these signals 
to the components as shown in FIG. 6. 
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To store processed image data into the memory card 22, 
the system controller 13A outputs required data and addresses 
to the memory card 22 via the input/output interface llg and 
the connector 30 of the camera proper llA and via the 
5 input/output interface 21 of the memory card attachment 20. 
The required data includes a file name necessary for image data 
input/output, a photographing date, a photographing condition 
(exposure value , etc.) , and an image processing condition (gain 
adjusting condition) . 

10 

The data input unit llhA includes components for user 
operation such as a mode selector button, a shutter/release 
button, and a liquid crystal monitor on which the mode of the 
electronic still camera lOA is displayed, not shown in the 
15 figure. When the user manipulates the mode selector button 
or the shutter release button, the data input unit llhA outputs 
operational result data to the system controller 13A. 

User instructions entered by the mode selector button 
20 include whether to perform gain adjusting and, if gain 
adjusting is to be performed, whether to perform white balance 
correction and/or brightness correction. In addition, the 
number of times the parameters for each of white balance 
correction and brightness correction are to be changed may be 
25 specified. Whether to perform black level of f setting may also 
be specified. 

When the system controller 13A receives, before gain 
adjusting, an instruction which allows the electronic still 
30 camera lOA to automatically decide the number of frames, the 
system controller 13A decides the number of frames that is set 
as the default value , for example , five frames for white balance 
correction and four frames for brightness correction. 
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In this embodiment, five types of parameters are provided 
for white balance correction, and four types for brightness 
correction. More specifically, five color temperature 
parameters are provided (corresponding to daytime light, 
5 morning and evening light, cloudy-day light, incandescent 
light, and fluorescent light) . 

There are four types of brightness correction. More 
specifically, the exposure values for the highlight area are 

10 0.0, -0.5, -1.0, and -1 . 5EV, and four exposure values for the 
shadow area are 0.0, +0.5, +1.0, and +1.5EV. The exposure 
values for the highlight area, 0.0, -0.5, -1.0, and -1.5EV, 
are combined with the exposure values for the shadow area , +1.5, 
+1.0, +0.5, and O.OEV. That is, the exposure value of -1 . 5EV 

15 for the highlight area is combined, for example, with the 
exposure value of 0 . OEV f or the shadow area. It is to be 
understood that the present invention is not limited to those 
specific combinations. 

20 When the user specifies the number of frames, the 

parameters are selected automatically from the above- 
described five types and four types. For example, if the user 
selects three types for white balance correction and two types 
for brightness correction, then daytime light, cloudy-day 

25 light, and incandescent light are selected as the white balance 
parameters . As the brightness correction parameters ,-0.5 and 
-1 . OEV are selected for the highlight area , and +1.0 and +0 . 5EV 
for the shadow area . 

30 The signal processor 212 , adapted for processing digital 

signals, may be implemented in the form of a digital signal 
processing IC. Because the primary operational function of 
the processor 212 is multiplication and addition, a 
general-purpose DSP may also be used. 
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Next, the area extractor 212a and the gain adjusting 
circuit 212c of the signal processor 212, which has the 
components described above, will be described more in detail. 

5 First, the area extractor 212a is adapted to find out 

the highlight areas and the shadow areas included in one frame 
of image and outputs area information indicating which area 
each pixel belongs to . There are many area extraction methods . 
The present invention is not limited to any particular area 

10 extraction method. In the description of the alternative 
embodiment, an application will be described in which the edge 
detection is combined with the area division. With such an 
application, the area extractor 212a comprises a luminance 
signal generator 300, an edge detector 302, and an area divider 

15 304 interconnected as shown in FIG. 9. 

The- luminance signal generator 300 is adapted to generate 
the luminance signal Y of each pixel based on the color data 
(RGB data) received from the buffer memory 12b. The luminance 
20 signal Y and RGB data are related as indicated by expression 
(2) shown below. The luminance signal generator 300 generates 
the luminance signal Y based on this expression. The areas 
are extracted based on the obtained luminance signal Y. 

25 Y = 0.30R + 0.59G + O.llB (2) 

Areas are extracted in two stages. First, the edge 
detector 302 detects an edge. The edge detector 302 checks 
the luminance signal Y to detect a point at which the brightness 
30 of the image changes, and finds out a candidate for the boundary 
between the highlight and shadow areas. 

More specifically, an edge in a boundary between the 
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highlight and shadow regions is a step-type edge whose 
brightness changes gradually. One method of detecting a 
step-type edge is to find out the spatial, primary derivative, 
df (x,y) , of the brightness signal, f (x,y) , of each pixel. If 
the absolute value of the derivative is larger than a 
predetermined value , then the edge detector 302 determines that 
the pixel constitutes a step-type edge. The expression (x,y) 
represents the spatial (horizontal and vertical) position of 
a pixel in the coordinates. For example, when one frame of 
image data is composed of 488 horizontal lines each including 
720 pixels, l<x^720 and lSy^488. The derivative df (x,y) is 
represented by the expression given below. 

df(x,y) = (5f/ax) + ( 5f/5y) (3) 

The absolute value of the derivative is as shown below. 

|df(x,y)| = [{df/d^)^+ {df/dy)^]'^^ (3) 

Next, the area divider 304 determines the highlight area 
and the shadow area. More specifically, the area divider 304 
divides the image according to the boundary candidate obtained 
by the edge detector 302 and, for each obtained area, finds 
out the number of pixels included in the area and the average 
of the brightness values of the pixels included in the area. 
With the number of pixels plotted on the vertical axis and the 
average of the brightness values of the pixels included in each 
area on the horizontal axis, the histogram shown in FIG. 10 
is created. This histogram typically has two peaks, for 
example, when the image is photographed against the sun. The 
peak of the higher brightness part corresponds to the highlight 
area, and the peak of the lower brightness area to the shadow 
area . 
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By means of the trough between the two peaks as the 
threshold, the area divider 304 divides the whole area of the 
image into two. Specifically, one area whose brightness is 
higher than the threshold is defined as the highlight area, 
5 while the other area whose brightness is lower than the 
threshold is defined as the shadow area. After defining the 
areas as described above, the area extractor 212a outputs to 
the area storage 212b the binary value '^1" for pixels included 
in the highlight area, and the binary value "0" for pixels 
10 included in the shadow area, in response to the address 
information supplied from the system controller 13A. 

The area storage 212b retains this area information until 
the next image is photographed and new area information is 
15 written in. The retained area information is output to the 
gain adjusting circuit 212c in response to an output 
instruction and address information from the system controller 
13A. 

20 The gain adjusting circuit 212c receives this area 

information from the area storage 212b and image data from the 
buffer memory 12b. From the area storage 212b and the buffer 
memory 12b, the image data is sent to the gain adjusting circuit 
212c in the raster scanning fashion. Specifically, one 

25 horizontal line of image data is output followed by the next 
one line of image data. One unit of image data from the buffer 
memory 12b includes 24 bits in length (8 bits for each R, G, 
and B data) , and is sent in parallel over eight signal lines, 
i.e. eight bits at a time. 

30 

FIG. 11 shows the exemplified configuration of the gain 
adjusting circuit 212c. The gain adjusting circuit 212c 
comprises the selector 123 , the gain control 124 , a black level 
control 1241, a gradation control (LUTs) 125A, the 
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luminance-chrominance matrix 126, the saturation emphasizing 
circuit 128, and a parameter setter 129A. 



Table 2 shows how the gain control 124 , gradation control 
5 125A, luminance-chrominance matrix 126, and saturation 
emphasizing circuit 128 convert image data during white balance 
correction and brightness correction described above. 



Table 2 





Bracketing effect 


White balance 
correction 


Brightness 
correction 


Gain 


Change Kr and Kb 




LUT 




Change 


Matrix coefficient 


Change coefficient 
for optimum color 
reproduction 




Saturation emphasis 


Change 





10 

The rows Gain, LUT, Matrix coefficient and Saturation 
emphasis in Table 2 represent how the parameters change during 
the processing of the gain control 124, gradation control 125A, 
luminance-chrominance matrix 126, and saturation emphasizing 
15 circuit 128, respectively. In Table 2, Kr and Kb represent 
the gain of R data and B data, respectively. 

The settings listed in Table 2 will be described. The 
reason for making the brightness correction to achieve a 

20 bracketing effect was described above. The reason for making 
the gain, matrix coefficient and saturation emphasis change 
to adjust a white balance will now be described. For 
photographic representation, the selection of light and color 
temperature is important. The color temperature varies with 

2 5 the photographing time and photographing location. The image 
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may be tinged with red or blue by changing the color temperature 
setting. This, in turn, changes the image impression. 

Therefore, an image photographed at some color 
5 temperature is processed at the color temperature setting 
appropriately selected so that the optimum image is reproduced. 
Changing the saturation also emphasizes the color. This is 
the reason for changing color temperature or saturation during 
white balance correction. For this reason, the gain, matrix 
10 coefficients, and saturation emphasis are changed as listed 
in Table 2 . 

One or two of the settings of the gain, matrix 
coefficients, and saturation emphasis may be selectively 
15 combined. 

White balance correction may be made in one of the several 
ways. For example, a plurality of cases are assumed with 
respect to the color temperature of the incident light so that 

2 0 the gain, matrix coefficients, and saturation emphasis 
settings are prepared in accordance with the respective cases 
of color temperature. The color temperatures to be assumed 
should include those for daytime light, morning and evening 
light, cloudy-day light, fluorescent light, and incandescent 

25 light. 

Another method is to change the gain , matrix coefficients , 
and saturation emphasis to maintain white balance regardless 
of the color temperature of the incident light. Any of the 
30 methods may be used in this invention. In the specific 
embodiment, the color temperatures are set. 

From the system controller 13A, the parameter setter 129A 
in FIG. 11 receives an instruction indicating how white balance 
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correction and brightness correction are to be made, as well 
as the parameters to be used for these corrections, and then 
sets up the components of the gain adjusting circuit 212c 
accordingly. 

5 

More specifically, for use in the gain control 124, the 
parameter setter 129A has five combinations of gains Kr and 
Kb (corresponding to gains of an amplifier) associated with 
five color temperatures. In response to an instruction from 

10 the system controller 13A, the parameter setter 129A selects 
one of the gain combinations and outputs the selected one to 
the gain control 124. In this specific embodiment, the gain 
Kg of G data is fixed. The gain control 124 multiples color 
data by the received gain. The parameter setter 129A also does 

15 the same processing for the luminance-chrominance matrix 126 
and the saturation emphasizing circuit 128. The setter 129A 
outputs the signal indicating a gradation correction curve to 
the gradation control 125A. The gradation correction curve 
will be described later. 

20 

Next, the configuration and the operation of the gain 
adjusting circuit 212c will be described. The selector 123 
distributes RGB data, sequentially sent from the buffer memory 
12b, into R data, G data, and B data. The gain control 124 

25 amplifies the data of each color distributed by the selector 
123. More specifically, the gain control 124 receives numeric 
values (Kr, Kb) predetermined according to the color 
temperature from the parameter setter 129A and multiples color 
data by the numeric values. The multiplication result will 

30 be forwarded to the gradation control 125A. 

The black level control 1241 receives the addition value 
(positive or negative integer) from the system controller 13A 
via the parameter setter 129A and performs the same addition 
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on each R, G and B color data. This processing is the same 
for all the pixels. 

The gradation control 125A converts gradation when it 
5 receives a brightness correction instruction. At this time, 
the gradation control 125A receives color data and area 
information on each pixel from the buffer memory 12b and the 
area storage 212b, respectively, and also receives an 
instruction indicating an exposure value from the parameter 

10 setter 129A. The gradation control 125A has two gradation 
correction curves , one for the highlight area and the other 
for the shadow area, indicating the correspondence between the 
values of color date sent to the gradation control 125A and 
the values of color data to be output. The gradation control 

15 125A converts the color data according to this correction curve . 
FIGS . 12A and 12B show examples of gradation correction curves . 
FIG. 12A plots gradation correction curves for the shadow area. 
The curves indicated by (1) , (2) , (3) , and (4) correspond to 
the exposure values of +1.5, +1.0, +0.5, and O.OEV, 

20 respectively. FIG. 12B plots gradation correction curves for 
the highlight area. The curves indicated by (1) , (2) , (3) , 
and (4) correspond to -1 . 5 , -1.0, -0.5, and O.OEV, respectively. 
In FIGS. 12A and 12B, the horizontal axis indicates the value 
of input color data, and the vertical axis indicates the output 

25 value of color data with respect to the input color data. 

The gradation control 125A selects one of the correction 
curves according to the criterion described below. A set of 
exposure values of the highlight and shadow areas, for example 
30 (-0.5, +1.0), is sent from the parameter setter 129A to the 
gradation control 125A. A gradation correction curve to be 
used is determined based on this information as well as 
information stored in the area storage 212b in the form of 
binary "1" and "0". Specifically, when the area information 
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is binary"!", the curve (3) corresponding to -0 . 5EV is selected 
from the correction curves shown in FIG. 12B. When the area 
information is binary "0", the curve (2) corresponding to 
+1 . OEV is selected from the correction curves shown in FIG. 
5 12A. The resultant color data, thus corrected in gradation, 
will in turn be output to the luminance-chrominance matrix 12 6 . 

The luminance-chrominance matrix 126 performs matrix 
operation to convert RGB data to luminance signal Y and color 
10 difference signals Cr and Cb for optimum color reproduction 
at each color temperature. Matrix operation is performed 
according to the expressions described below. 



Yl = Kll X Rl + K12 X Gl + K13 x Bl , 
15 Crl = K21 X Rl + K22 x Gl + K23 x Bl , (5) 

CBl = K31 X Rl + K32 x Gl + K33 x Bl, 



where, Rl , Gl , and Bl are color data that is input to the 
luminance-chrominance matrix 126. They represent the red, 

20 green and blue components, respectively. Kll, K12, K13, K21, 
K22, K23, K31, K32 , and K33 are conversion coefficients. Yl 
is the luminance signal that is output by the luminance- 
chrominance matrix 126. Crl and Cbl are color difference 
signals output by the luminance-chrominance matrix 126. The 

25 luminance-chrominance matrix 126 has five sets of conversion 
coefficients, each set corresponding to one of the color 
temperatures. The luminance-chrominance matrix 126 selects 
the conversion coefficients according to the specification of 
the color temperature from the parameter setter 129A to perform 

30 the calculation of expression (5) . 

The color difference signals Crl and Cbl are sent to the 
saturation emphasizing circuit 128. The saturation 

emphasizing circuit 128 receives five sets of coefficients each 
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corresponding to one of the five color temperatures from 
parameter setter 129A. The received coefficients are 
multiplied by the color difference signals Crl and Cbl . 



5 Image data thus processed by the gain adjusting circuit 

212c of the signal processor 212 is transferred to the data 
compressor llf shown in FIG. 7. The image data may be 
compressed, for example, according to the JPEG standard. The 
compressed image data is sent to the memory card 22 and stored 
10 therein. 

Next, the operation of the electronic still camera lOA 
with the configuration described above will be described. The 
user manipulates the data input unit llhA to specify whether 

15 to perform gain-adjusting. When the user wishes to do the gain 
adjusting, he or she must specify whether to make white balance 
and/or brightness corrections, whether the specification of 
the number of parameter changes is automatic and, if not 
automatic, the number of times the parameters are to be changed. 

20 When the user wants to do the black level offsetting, he or 
she must specify whether to do the offsetting. 

Upon receiving the user instruction, the system 
controller 13A decides the number of times the white balance 

2 5 correction parameters are to be changed and the number of times 
the brightness correction parameters are to be changed. When 
both white balance and brightness corrections are to be made, 
the system controller 13A tells the signal processor 212 to 
do the processing the number of times equal to the number of 

30 times white balance correction parameters are to be changed 
multiplied with the number of times brightness correct.ion 
parameters are to be changed. 

In the description below , the operation will be described 
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assuming that the user has specified that both white balance 
and brightness corrections are to be made, that the white 
balance correction parameters are to be changed three times, 
and that the brightness correction parameters are to be changed 
5 twice. In this case, six frames of image data will be finally 
obtained . 

When the user depresses the shutter release button after 
specifying the processing mode setting, the imaging device 11c 

10 photographs an image of an object. The obtained image data 
is processed by the preprocessor lid and the A/D converter lie 
and then sent to the signal processor 212. After color- 
interpolated by the color interpolation circuit 12a of the 
signal processor 212, the image data is stored in the buffer 

15 memory 12b. 

Since the instruction to perform the black level 
offsetting has been issued, the system controller 13A makes 
access to the buffer memory 12b to determine whether to perform 
20 addition or subtraction on the gradation values as described 
above. Before addition or subtraction, the system controller 
13A decides the addition value or subtraction value and outputs 
the decided value to the parameter setter 129A. 

25 The instruction to make brightness correction has been 

issued, so that the area extraction is performed. Each time 
an output instruction and an address are received from the 
system controller 13A, the image data is output from the buffer 
memory 12b to the area extractor 212a. The area extractor 212a 

30 extracts or distinguishes the highlight area and the shadow 
area from each other to store the obtained area information 
in the area storage 212b. 

After the area extraction is finished, the image data 
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is output from the buffer memory 12b to the gain adjusting 
circuit 212c each time an output instruction and an address 
are received from the system controller 13A. For the first 
three frames of image data, the gain adjusting circuit 212c 
5 uses -0.5EV for the highlight area and +1 . OEV for the shadow 
area. For the last three frames of image data, the gain 
adjusting circuit 212c uses -1 . OEV for the highlight area and 
+0.5EV for the shadow area. 

10 The processing shown in Table 2 is repeated in the gain 

control 124, gradation control 125A, luminance-chrominance 
matrix 126, and saturation emphasizing circuit 128 of the gain 
adjusting circuit 212c six times while varying the parameters . 
The black level offsetting is also repeated six times in the 

15 black level adjuster 1241. 

The image data thus processed by the gain adjusting 
circuit 212c is sent to the data compressor llf for compression. 
After that, the image data is sequentially stored in the memory 
2 0 card, one frame at a time. 

Unlike an auto-bracketing camera according to the prior 
art, the electronic still camera according to the present 
invention shoots an object to capture an image thereof only 

2 5 once. Therefore, the problem that the photographing timing 

differs with photographed images does not arise. As a result, 
the user may obtain a desired image without concerning a 
movement of the object, the changes of the photographing 
conditions, or an unintentional movement of the hands of the 

3 0 operator during photographing. 

The illustrative embodiments of the present invention 
described above differ from each other only in the 
configuration of the buffer memory and the subsequent 
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components of the signal processor. Because the differing 
components may be implemented in software, one and the same 
piece of hardware may be used to implement both of the 
illustrative embodiments when the signal processor is built 
5 with a CPU or a DSP. That means that both of the illustrative 
embodiments may be provided in an electronic still camera 
simply with the software thereof modified accordingly. 

As described above, the image signal processor according 
10 to the present invention gives the user the effect, equivalent 
to auto-bracketing according to the prior art, by simply 
photographing an object only once. In addition, a plurality 
of images may be produced with their own gradation 
characteristics, color temperature, saturation, contour 
15 enhancement, data compression ratio and black level changed 
appropriately. Therefore, the problem that the photographing 
timing differs with photographed images is not involved. As 
a result, the user may obtain a desired image without taking 
accout of the movement of an object, changes fo the 
20 photographing conditions, or an unintentional movement of the 
user's hands during photographing. 

The processor and the method according to the present 
invention allow the brightness, gradation characteristics, 
25 color temperature, saturation, contour enhancement, data 
compression ratio, and black level of an original image to be 
changed, thus making it possible to create various images that 
meet various purposes from one photographed image. 

3 0 The entire disclosure of Japanese Patent Application No . 

228950/1999 filed on August 13, 1999 including the 
specification, claims, accompanying drawings and abstract of 
the disclosure is incorporated herein by reference in its 
entirety . 



While the present invention has been described with 
reference to the particular illustrative embodiments, it is 
not to be restricted by those embodiments but only by the 
appended claims. It is to be appreciated that those skilled 
5 in the art can change or modify the embodiments without 
departing from the scope and spirit of the present invention. 
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WHAT IS CLAIMED IS: 

1. An image signal processor for performing image 
processing on a first image signal representative of an image 
of a subject field captured by an imaging device to produce 
a second image signal, comprising: 

5 a first memory for storing therein the first image 

signal; 

a plurality of image processors for each performing image 
processing, different from each other, on the stored first 
image signal to produce the second image signal different from 
10 each other; and 

a second memory for storing therein the second image 
signals produced, 

wherein said plurality of image processors include types 
and parameters of the image processing such that at least one 
15 of the types and parameters of the image processing are 
different between said plurality of image processors. 

2. The image signal processor according to claim 1 
wherein the types of image processing performed by said 
plurality of image processors include at least one selected 
from a group consisting of a change of brightness, a change 

5 of gradation change characteristics, a correction of a color 
temperature, a change of saturation, a change of a contour, 
a change of a compression ratio and a change of a black level 
of the first image signal stored in said first memory, 

the parameters of image processing being of the at least 
10 one selected from the group, 

said plurality of image processors performing the image 
processing of the at least one selected from the group on the 
first image signal stored in said first memory according to 
the parameters of image processing. 
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3. The image signal processor according to claim 1 
wherein each of said plurality of image processors corresponds 
to one of a plurality of display units which are provided for 
visualizing the image represented by the second image signals 
stored in said second memory, 

each of said plurality of image processors processing, 
according to the parameters, the second image signals to be 
displayed on one of the display units which corresponds to said 
image processor. 

4. The image signal processor according to claim 3 
wherein the plurality of display units include a CRT (Cathode 
Ray Tube) display and an image printer. 

5. The image signal processor according to claim 2, 
further comprising a divider circuit for dividing the first 
image signal stored in said first memory into a highlight area 
and a shadow area , 

each of said plurality of image processors performing 
the image processing in which at least one of the types and 
the parameters of the image processing differs between the 
highlight area and the shadow area. 

6. The image signal processor according to claim 1 
wherein said second memory is detachably connected to said 
image signal processor. 

7. An image signal processor for performing image 
processing on a first image signal representative of an image 
of a subject field captured by an imaging device to produce 
a second image signal, comprising: 

a first memory for storing therein the first image 
signal; 

a plurality of image processors for each performing image 



processing, different from each other, on the stored first 
image signal to produce a third image signal different from 

10 each other; 

a second memory for storing therein the third image 
signals produced, and 

an image composer circuit for composing the third image 
signals to produce the second image signal, 

15 wherein said plurality of image processors include types 

and parameters of the image processing such that at least one 
of the types and parameters of the image processing are 
different between said plurality of image processors. 

8. The image signal processor according to claim 7, 
further comprising a divider circuit for dividing the first 
image signal stored in said first memory into a highlight area 
and a shadow area, 

5 wherein said plurality of image processors are provided 

correspondingly to the highlight area and the shadow area, each 
of said plurality of image processors performing the image 
processing for one of the highlight area and the shadow area 
to produce the third image signals. 

9. A method of processing a first image signal 
representative of an image of a subject field captured by an 
imaging device to produce a second image signal, comprising 
the steps of: 

5 storing the first image signal in a memory; 

performing a same type of image processing on the stored 
first image signal according to parameters of image processing 
different from each other to produce the second image signals; 
and 

10 storing the produced second image signals in a memory. 

10. Imaging apparatus comprising: 
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an imaging device for capturing an image of a subject 
field and producing a first image signal representative of the 
subject field; 

5 a first memory for storing therein the first image 

signal ; 

a plurality of image processors for each performing image 
processing, different from each other, on the stored first 
image signal to produce the second image signal different from 
10 each other; 

a second memory for storing therein the second image 
signals produced; and 

an image composer circuit for composing the second image 
signals to produce a third image signal, 
15 said plurality of image processors including types and 

parameters of the image processing such that at least one of 
the types and parameters of the image processing are different 
between said plurality of image processors, 

the types of image processing including at least one 
20 selected from a group consisting of a change of brightness, 
a change of gradation change characteristics, a correction of 
a color temperature, a change of saturation, a change of a 
contour, a change of a compression ratio and a change of a black 
level of the first image signal stored in said first memory, 
25 the parameters of image processing being of the at least 

one selected from the group, 

whereby said plurality of image processors perform the 
image processing of the at least one selected from the group 
on the first image signal stored in said first memory according 
30 to the parameters of image processing. 
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ABSTRACT OF THE PROCEDURE 
A signal processor of an electronic still camera stores 
image data, obtained during photographing , into a buff er memory 
A gain adjusting circuit performs multiplication on the stored 
5 image data with the exposure amount changed by 0 . 5EV at a time . 
Next, a usage-specific processor processes the image data in 
one of two ways to change the gradation, contour, and so on, 
of the image data so that the image data may be output 
appropriately to a CRT or a printer. 
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DATE' 

July 26, 2000 


Residence (City, State & Country) 

Asaka-shi, Saitama, Japan 


CITIZENSHIP 

Japanese 


CComniBte.Street Address including City. State & Country) 

C/O Fu3i PTioto Tilm Co., Ltd., 11-46, Senzui 3-chome, 
Asaka-shi, Saitama, Japan 


GIVEN NAME FAMILY NAME 


INVENTOR'S SIGNATURE 


DATE* 


Residence (City, State & Country) 


CITIZENSHIP 


POST OFFICE ADDRESS (Complete Street Address including City, State & Country) 


GIVEN NAME FAMILY NAME 


INVENTOR'S SIGNATURE 


DATE* 


Residence (City, State & Country) 


CITIZENSHIP 


POST OFFICE- ADDRESS (Complete Street Address including City, State & Country) 



* DATE OF SIGNATURE 



